None exec_jwst_background_subtraction_miri_imaging_testing

JWST Pipeline Validation Notebook: calwebb_image2, background subtraction for MIRI imaging

**Instruments Affected**: e.g., FGS, MIRI, NIRCam, NIRISS

Table of Contents


[Introduction\*](#intro)
[JWST CalWG Algorithm\*](#algorithm)
[Defining Terms](#terms)
[Test Description\*](#description)
[Data Description\*](#data_descr)
[Imports\*](#imports)
[Loading the Data\*](#data_load)
[Run the Pipeline](#pipeline)
[Passing criteria](#testing)
[About This Notebook\*](#about)

Introduction

This is the validation notebook for the background subtraction step as part of calwebb_image2. This step takes in a set of images and a set of background observations. If more than one background observation is given, they are combined into a sigma clipped mean before being subtracted from each of the science data images. For more information on the pipeline step visit the links below.

Step description: https://jwst-pipeline.readthedocs.io/en/latest/jwst/background_step/description.html

Pipeline code: https://github.com/spacetelescope/jwst/tree/master/jwst/background

Top of Page

JWST CalWG Algorithm

The page describing the algorithm and any details can be found here:

https://outerspace.stsci.edu/display/JWSTCC/Vanilla+Imaging+Background+Subtraction

Top of Page

Defining Terms

Here are some common terms that will be used throughout the notebook

JWST: James Webb Space Telescope

MIRI: Mid-Infrared Instrument

MIRISim: Simulator that creates pipeline ready MIRI simulated data

Top of Page

Test Description

This test is performed by creating a set of simulated data with multiple point sources located at random locations across the MIRI field of view. There is a set of 4 simulated images at four dithered positions with 300+ stars and a galaxy in the MIRI field of view (including Lyot). This test also takes in a set of four simulated background images, at four dithered positions, with around 40 fainter stars across the field of view and Lyot mask. All images will be processed through calwebb_detector1, and put into an association file to be run through calwebb_image2. This will tell the background step which are the science observations and which are the background observations so that it will do a sigma clipped mean of the background exposures, then subtract the mean from each of the science observations.

The notebook shows the images (background, science, averaged background, and background subtracted) through the course of the notebook to demonstrate how well the algorithm works.

Top of Page

Data Description

The set of data used in this particular test were created with the MIRI Data Simulator (MIRISim). There is a set of 4 simulated images at four dithered positions with 300+ stars and a galaxy in the MIRI field of view (including Lyot). This test also takes in a set of four simulated background images, at four dithered positions, with around 40 fainter stars across the field of view and Lyot mask. All of the images were created with the F770W filter.

Top of Page

Set up Temporary Directory

The following cell sets up a temporary directory (using python's tempfile.TemporaryDirectory()), and changes the script's active directory into that directory (using python's os.chdir()). This is so that, when the notebook is run through, it will download files to (and create output files in) the temporary directory rather than in the notebook's directory. This makes cleanup significantly easier (since all output files are deleted when the notebook is shut down), and also means that different notebooks in the same directory won't interfere with each other when run by the automated webpage generation process.

If you want the notebook to generate output in the notebook's directory, simply don't run this cell.

If you have a file (or files) that are kept in the notebook's directory, and that the notebook needs to use while running, you can copy that file into the directory (the code to do so is present below, but commented out).

Top of Page

If Desired, set up CRDS to use a local cache

By default, the notebook template environment sets up its CRDS cache (the "CRDS_PATH" environment variable) in /grp/crds/cache. However, if the notebook is running on a local machine without a fast and reliable connection to central storage, it makes more sense to put the CRDS cache locally. Currently, the cell below offers several options, and will check the supplied boolean variables one at a time until one matches.

If the above cell (creating a temporary directory) is not run, then setting crds_cache_tempdir to True will store the CRDS cache in the notebook's directory (the same as setting crds_cache_notebook_dir to True).

Imports

List the package imports and why they are relevant to this notebook.

Top of Page

Loading the Data

Download data from Box to use in the notebook.

Top of Page

Run calwebb_detector1

Top of Page

Look at rate images

Display the rate science images and background images to see locations of sources.

Create a Level2 association file of the science and background exposures

Run association file through background subtraction step of calwebb_image2

The default value of sigma for the background subtract step is set to 3, but may need to be adjusted downward to 2 or 1 in order to actually sigma clip the sources in the images. Test this for your data. For this particular data set, 2 is sufficient, but for brighter sources, 1 may be the best option.

Look at averaged background image

See how well the sigma clipping did at removing the sources from the background image. If the sources in the background image are bright, the value of sigma should be set to 1. If the sources are faint enough, the default value of 3 should be good enough.

Also look at the value of a specific pixel in the averaged image, one that has a source in at least one of the background images, to see if the flux was adequately removed in the sigma clipping process.

Look at the values of pixels in the background images at the location of one of the stars to see whether the star flux is being rejected as part of the sigma clipping. In calwebb_image2, the sigma value was set to 2, which is lower than the default value of 3. This allows the most pixels to be rejected as outliers, and should leave only the background values in the final averaged image.

Look at background subtracted data

Passing criteria

Examine the images shown and the pixel values reported through the notebook. If the averaged background image is subtracted from the science images (subtracted background values near 0), and the averaged background image shows a smooth background with the sources removed, then this test passes. The four background images should be averaged together after the sources were rejected via sigma clipping. Check that the subtracted background values are near 0.

Top of Page

About this Notebook

Author: M Cracraft, Senior Staff Scientist, INS/MIRI branch
Updated On: 06/09/2021

Top of Page stsci_pri_combo_mark_horizonal_white_bkgd